2 research outputs found
Adapting Language Models for Non-Parallel Author-Stylized Rewriting
Given the recent progress in language modeling using Transformer-based neural
models and an active interest in generating stylized text, we present an
approach to leverage the generalization capabilities of a language model to
rewrite an input text in a target author's style. Our proposed approach adapts
a pre-trained language model to generate author-stylized text by fine-tuning on
the author-specific corpus using a denoising autoencoder (DAE) loss in a
cascaded encoder-decoder framework. Optimizing over DAE loss allows our model
to learn the nuances of an author's style without relying on parallel data,
which has been a severe limitation of the previous related works in this space.
To evaluate the efficacy of our approach, we propose a linguistically-motivated
framework to quantify stylistic alignment of the generated text to the target
author at lexical, syntactic and surface levels. The evaluation framework is
both interpretable as it leads to several insights about the model, and
self-contained as it does not rely on external classifiers, e.g. sentiment or
formality classifiers. Qualitative and quantitative assessment indicates that
the proposed approach rewrites the input text with better alignment to the
target style while preserving the original content better than state-of-the-art
baselines.Comment: Accepted for publication in Main Technical Track at AAAI 2